Policy-Guided Heuristic Search with Guarantees

The use of a policy and heuristic function for guiding search can be quite effective in adversarial problems, as demonstrated by AlphaGo its successors, which are based on the PUCT algorithm. While also used to solve single-agent deterministic it lacks guarantees effort computationally inefficient practice. Combining A* algorithm with learned tends work better these domains, but variants do not policy. Moreover, purpose using is find solutions minimum cost, while we seek instead minimize loss (e.g., number steps). LevinTS guided provides steps that relate quality policy, does make function. In this introduce Policy-guided Heuristic Search (PHS), novel uses both has theoretical relates We show empirically sliding-tile puzzle, Sokoban, puzzle from commercial game `The Witness' PHS enables rapid learning compares favorably A*, Weighted Greedy Best-First Search, LevinTS, terms problems solved time all three domains tested.